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1 0 Background of the Invention 

1. Technical Field. 

The invention relates in general to digital image processing, and in particular to a system 
for minimizing the visual effect of local pixel defects. 

2. Related Art 

15 Pixel defects are a well-known problem in digital photography. A typical digital imager 

has an array of hundreds of thousands to millions of photocells (i.e., pixels) that produce 
electrical responses to light exposure. Ideally, each pixel should produce the same electrical 
response for a given quantity (i.e., intensity and the duration) of light exposure. Usually, 
however, at least a few of those pixels will be defective. Some isolated photo cells or pixels may 

20 produce too much, too little, or even no electrical response at all. As a result, a captured image 
may have bright or dark point defects. 

One method for minimizing pixel defects in an image is to transform the image from the 
spatial domain to the spatial frequency domain, apply various filters to remove the high 
frequency components which tend to be associated with point defects, and then convert the 

25 image back to the spatial domain. This method of eliminating pixel defects utilizes a great deal 
of processing power, without utilizing the real-time, cost-effective Fourier-transforming 
capability of a simple optical device. 
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Summary 



An imaging system is provided to minimize pixel defects by capturing an image in the 
spatial frequency domain. The imaging system comprises an imager having an array of 
* photocells. Each photocell produces an electrical response to light exposure. The electrical 
5 responses of the array of photocells together represent spatial frequency-domain image data. An 
image processor receives the spatial frequency-domain image data from the imager and 
transforms the spatial frequency-domain image data into spatial-domain image data. The 
imaging system further comprises a filter that detects and removes noise from the spatial 
frequency-domain image data before it is transformed into spatial-domain image data. The 
10 imaging system further comprises a user interface that displays the spatial-domain image data. 

In another imaging system embodiment, an optical lens is placed between a spatial 
representation of an image object and the imager. The optical lens performs an approximate 
Fourier transform on light emanating from the spatial representation of the image object toward 
the imager. An image processor performs an approximate discrete inverse Fourier transform on 
15 the data received from the imager, restoring the spatial representation of the image. 

In another imaging system embodiment, a focusing lens and transform lens are placed 
between the imager and an image object. The focusing lens focuses light onto an image plane 
between the focusing lens and the transform lens. A transform lens receives the light from the 
focusing lens and bends it to form a diffraction pattern of the image object at a transform plane 
20 of the transform lens. The imager is placed at the transform plane of the transform lens to 
capture the diffraction pattern of the image object 

The invention also provides a methodology that minimizes point defects in an image. 
First, a transform lens is placed between an image object and the digital imager. The transform 
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lens performs an approximate Fourier transform on light traveling between the object and the 
digital imager. Then, the digital imager captures a diffraction pattern of an image object and 
produces digital spatial frequency-domain image data corresponding to the captured diffraction 
* pattern of the object. The spatial frequency-domain image data is transferred to an image 
5 processor, which performs an approximate discrete inverse Fourier transform on the spatial 
frequency-domain image data, converting it to a spatial domain. Finally, the spatial-domain 
image data is stored in digital memory or displayed. 

Other systems, methods, features and advantages of the invention will be or will become 
apparent to one with skill in the art upon examination of the following figures and detailed 
10 description. It is intended that all such additional systems, methods, features and advantages be 
included within this description, be within the scope of the invention, and be protected by the 
accompanying claims. 

Brief Description of the Figures 
15 The components in the figures are not necessarily to scale, emphasis instead being placed 

upon illustrating the principles of the invention. Moreover, in the figures, like reference 
numerals designate corresponding parts throughout the different views. 

FIG. 1 is a part perspective, part block diagram of an imaging system that minimizes 
pixel defects by capturing an image in the spatial frequency domain. 
20 FIG. 2 is a perspective diagram illustrating a frequency-domain image capture setup of 

the imaging system of FIG. 1. 

FIG. 3 is a perspective diagram illustrating another frequency-domain image capture 
setup of the imaging system of FIG. 1. 



3 



# • 

FIG. 4 is a functional flow diagram of a method for capturing an image that minimizes 
pixel defects. 

Detailed Description 

5 FIG. 1 is a diagram of an imaging system 100 that minimizes pixel defects by capturing 

an image in the spatial frequency domain. The imaging system 100 includes a frequency-domain 
image-capture setup 110, an image processor 120, and a user interface 130The frequency- 
domain image-capture setup 110 comprises an image object 111 (illustrated by a cup), a focusing 
lens 112 for focusing the image object 111, and a transform lens 114 that forms a diffraction 

1 0 pattern of the image object 111 onto an imager 118. The imager 118 may comprises an array of 
complimentary metal oxide semiconducting ("CMOS") photocells and circuit elements, captures 
the diffraction pattern of the image object 111. The imager 118 may also comprise a charged 
couple device ("CCD"). Each photocell produces an electrical response to light exposure. The 
electrical responses of the array of photocells together represent the spatial frequency-domain 

15 image data of the image object 111. The spatial frequency-domain image data is transferred 
from the imager 1 1 8 to the image processor 120. 

The image processor 120 comprises a controller 122 that controls the imager 118, a filter 
124 that filters out noise in the image, Fourier transform circuitry 126 that transforms the spatial 
frequency-domain image data back into the spatial domain, and memory storage 128 that stores 

20 the image data. The controller 122 sets the frame rate and length of exposure to which the 
imager is exposed to an image or sequence of images. The Fourier transform circuitry 126 
carries out standard digital processing techniques for computing discrete inverse Fourier 
transforms, well-known to those of ordinary skill in the art, to transform the captured spatial 
frequency-domain image data back into the spatial domain. The filter 124 optionally need not be 
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present in the image processor 120, but is incorporated into the illustrated embodiment to 
enhance the noise and artifact-minimizing features of the present invention. The memory storage 
128 comprises any suitable medium for storing digital data, such as random-access memory, 
» flash memory, hard disk memory, optical memory storage, and floppy disk memory. 
5 The user interface 130 comprises image capture setting controls 132, a display 134 and 

an image rendering setting control 136. The image capture setting controls 132 enables a user to 
fix the frame rate or the length of exposure of the imager 118 to an image. The image capture 
setting controls 132 also permit, but optionally need not permit, the user to control lighting 
conditions, aperture size, and the relative physical positioning of the focusing lens 112, the 

10 transform lens 114, and the imager 1 1 8. 

The display 134 displays the image after the image processor 120 transforms the spatial 
frequency-domain image data to the spatial domain. The display 134 also displays, but 
optionally need not display, the diffraction pattern represented by the spatial frequency-domain 
image data. The image rendering setting controls 136 permits a user to modify parameters that 

15 affect the filtering of the image data or the transformation of the image data from the spatial 
frequency to the spatial domains. . 

By capturing an image in the spatial frequency domain, spatial information about the 
image object is spread across multiple photocells. Moreover, each photocell is exposed to light 
emanating from several different locations on the image object. The effect of a defective 

20 photocell is distributed throughout the image and thus minimized Furthermore, the transform 
lens 114 performs an approximate real-time Fourier transform of the image data emanating from 
the image object, without utilizing any digital processing power. 
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FIG. 2A is a perspective diagram illustrating a frequency-domain image-capture setup 
210 of the imaging system 100 of FIG. 1. The image-capture setup 210 of the imaging system 
100 comprises an image object 211, a focusing lens 212, a transform lens 214, and an imager 
* 218. The imager 218 could comprise charge-coupled device components instead of, or in 
5 addition to, a complimentary metal oxide semiconductor image sensor. Also, the imager 218 
may by part of a still camera or a video camera. Furthermore, the imager 218 may be part of the 
same multi-chip module package or semiconducting wafer as the image processor 120, or 
alternatively be located remotely from the image processor 120. 

Light emanating from (or reflecting off) the image object 211 enters the focusing lens 
10 212. The light is coherent, but optionally need not be coherent. The focusing lens 212 focuses 
, light from the image object 211 into an image at an image plane 213. The focused light is 
illustrated by light rays 215. A transform lens 214 is positioned on the other side of the image 
plane 213. The transform lens 214 is separated from the image plane 213 by a distance equal to 
the focal length of the transform lens 214. Light rays 216 travel from the image plane 213 to the 
15 transform lens 214, where they are bent. Bent light rays 217 then travel from the image plane 
214 to the imager 218, that is positioned in the transform plane of the transform lens 214. At this 
position, the bent light rays 217 forma diffraction pattern of the image object 21 1 . 

FIG. 3 is a perspective diagram illustrating a second frequency-domain image-capture 
setup 230 of the imaging system 100 of FIG. L The image-capture setup 230 of the imaging 
20 system 100 comprises an image object 231, focusing lenses 233 and 243, first and second 
imagers 235 and 252, a laser 241, a spatial filter 244, a collimating lens 246, a spatial light 
modulator 248, and a transform lens 250. 
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The first imager 235 captures an image of the image object 231 in the spatial domain, 
much the same way as any conventional camera does. The image object 231 is illuminated by an 
incoherent light source, that reflects divergent light rays 232 toward the focusing lens 233. The 
* focusing lens 233 converts the divergent light rays 232 into converging rays 234. The 
5 converging rays 234 are captured by the first imager 235, which is located in the focal plane of 
the focusing lens 233. The first imager 235 communicates spatial-domain image data about the 
image object 23 1 to the spatial light modulator 248. 

The laser 241 produces a relatively straight beam 242 of coherent light. The focusing 
lens 243 may be a short-focal-length convex lens capable of transforming the straight beam 242 
10* into a divergent beam 245. The purpose of the focusing lens 243 is to widen the laser beam 
while maintaining an approximate uniformity of light intensity across the cross section of the 
laser beam. Before the beam 245 diverges, it passes through the spatial filter 244. The spatial 
filter 244, which comprises a pinhole placed at the focus of the "focusing lens 243, cleans up the 
laser beam 242. 

15 The collimating lens 246 converts the divergent beam 245 into a collimated (i.e., parallel) 

beam 247 of coherent light. The collimated beam 247 is directed toward the spatial light 
modulator 248, that is located in the input focal plane of the transform lens 250. The spatial light 
modulator 248 may comprise light-modulating components familiar to those of ordinary skill in 
the art, causes diffraction of the collimated beam 247, as would a two-dimensional transparency 

20 placed in the same focal plane. 

Diffracted light rays 249 emanating from spatial light modulator 248 are transformed by 
the transform lens 250 into a diffraction pattern at the transform plane (i.e., the output focal 
plane) of the transform lens 250. The second imager 252 may be placed in the transform plane 
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making it capable of capturing the diffraction pattern of the image object 231 and transmitting 
the data to an image processor (not shown).The frequency-domain image-capturing setup 230 
provides a means of capturing an image of an object illuminated by an incoherent light source in 
* the spatial frequency domain. 
5 FIG. 4 is a functional flow diagram of a method for capturing an image that minimizes 

pixel defects. A transform lens 310, such as a Fourier optic, is placed between an image object 
and an imager. The transform lens has an input focal plane and an output transform plane. The 
image object, or a spatial image representation is placed in the input focal plane of the transform 
lens, and the imager is placed in the transform plane of the transform lens. The transform lens 

10 device performs an approximate Fourier transform 320 on light traveling between from the 
image object to the imager. The imager captures a diffraction pattern 330 of the image object 
(i.e., captures an image in the spatial frequency domain) and produces spatial frequency-domain 
image data corresponding with the diffraction pattern] The imager transfers the spatial 
frequency-domain image data to an image processor 340. The image processor detects and 

15 filters out noise detected in the spatial frequency-domain image data 350. The image processor 
converts the spatial frequency-domain image data into spatial-domain image data 360 using 
standard discrete inverse Fourier transform techniques familiar to those of ordinary skill in the 
art. The spatial-domain image data is stored 370 in memory or displayed on a screen, monitor, 
or other suitable displaying device. 

20 Of course, some of the actions noted in FIG. 4 may be modified or omitted, and other 

actions not noted added, without departing from many of the inventive aspects of the invention. 
Moreover, the actions noted in FIG. 4 are not necessarily presented in any strict order. 
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While various embodiments of the application have been described, it will be apparent to 
those of ordinary skill in the art that many more embodiments and implementations are possible 
that are within the scope of this invention. Accordingly, the invention is not to be restricted 
* except in light of the attached claims and their equivalents. 
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